{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3852b9f0",
   "metadata": {},
   "source": [
    "# Chapter-4 微调大模型\n",
    "\n",
    "本节学习大纲\n",
    "\n",
    "- 理解大模型微调方法的原理与作用\n",
    "- 掌握使用 transformers + peft 框架进行大模型微调的基本流程\n",
    "\n",
    "在上一节课，我们深入学习了如何通过组织提示词激发 LLM 能力以解决复杂任务。通过组织提示词使 LLM 完成上下文学习来解决某些特定任务是使用 LLM 最高效、便捷的方式，但在某些特定场景下，存在仅通过提示词无法较好完成的任务，例如：\n",
    "\n",
    "- 高度自定义的复杂任务，较难通过有限的上下文让 LLM 充分理解任务要求；\n",
    "- 具有较高门槛的垂直领域任务，通用 LLM 不具备充分的领域知识，难以实现专业程度解决；\n",
    "- 对响应时间、算力要求较高的任务，无法使用大体量 LLM，需要用较小的 LLM 完成高难度任务。\n",
    "\n",
    "在这样的场景下，我们需要基于目标任务对通用 LLM 进行微调，从而使 LLM 充分学习目标任务，在特定的下游任务上达到较好的表现。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de53995b-32ed-4722-8cac-ba104c8efacb",
   "metadata": {},
   "source": [
    "## 4.1 环境配置\n",
    "\n",
    "本章内容将使用 datasets、transformers、peft 等框架完成从微调数据构造到高效微调模型的整体微调流程。首先，我们进行本章的环境配置。\n",
    "\n",
    "**注：由于在第二章已讲解本地如何部署并调用大模型，本章内容在第二章搭建环境基础上进行，请同学们首先完成第二章本地部署大模型的环境配置。**\n",
    "\n",
    "本章需要安装的第三方库如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0e9f2d16",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "!pip install -q datasets pandas peft"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1353a856",
   "metadata": {},
   "source": [
    "同样，本章使用第二章已下载的 Qwen-3-4B 模型作为微调的基座模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4012ab49",
   "metadata": {},
   "source": [
    "## 4.2 微调数据集构造"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "862ba7ea",
   "metadata": {},
   "source": [
    "### 4.2.1 什么是有监督微调（SFT）\n",
    "\n",
    "目前我们所使用的 LLM 一般经过了预训练、有监督微调、人类反馈强化学习三个步骤的训练。预训练是 LLM 强大能力的根本来源，事实上，LLM 所覆盖的海量知识基本都是源于预训练语料。但是，预训练赋予了 LLM 能力，却还需要第二步将其激发出来。经过预训练的 LLM 好像一个博览群书但又不求甚解的书生，对什么样的偏怪问题，都可以流畅地接出下文，但他偏偏又不知道问题本身的含义，只会“死板背书”。因此，我们还需要第二步来教这个博览群书的学生如何去使用它的知识，也就是 SFT（Supervised Fine-Tuning，有监督微调）。所谓有监督微调，其实就是将输入和输出同时给模型，让他根据输出不断去拟合从输入到输出的逻辑，类似于将问题和答案同时给模型，让模型基于答案学习解决问题的过程。在传统 NLP 时代，我们一般会针对每一个任务对模型进行微调。例如，一个经典的 NLP 任务可能是情感分类，也就是判断一句输入文本是积极情绪还是消极情绪，那么就会构造很多输入文本和其情感判断的数据，让模型去学会如何判断输入文本的情感。\n",
    "\n",
    "而面对能力强大的 LLM，我们往往不再是在指定下游任务上进行微调，而是选择训练模型的“通用指令遵循能力”，也就是一般通过`指令微调`的方式来进行 SFT。所谓指令微调，即我们训练的输入是各种类型的用户指令，而需要模型拟合的输出则是我们希望模型在收到该指令后做出的回复。例如，我们的一条训练样本可以是：\n",
    "\n",
    "    input:告诉我今天的天气预报？\n",
    "    output:根据天气预报，今天天气是晴转多云，最高温度26摄氏度，最低温度9摄氏度，昼夜温差大，请注意保暖哦\n",
    "\n",
    "也就是说，SFT 的主要目标是让模型从多种类型、多种风格的指令中获得泛化的指令遵循能力，也就是能够理解并回复用户的指令。因此，我们使用指令数据对模型进行 SFT 时，需要告知模型用户的指令和模型遵循这个指令所应该的输出。一般 SFT 所使用的指令数据集包括以下三个键：\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"instruction\":\"即输入的用户指令\",\n",
    "    \"input\":\"执行该指令可能需要的补充输入，没有则置空\",\n",
    "    \"output\":\"即模型应该给出的回复\"\n",
    "}\n",
    "```\n",
    "\n",
    "例如，如果我们的指令是将目标文本“今天天气真好”翻译成英文，那么该条样本可以构建成如下形式：\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"instruction\":\"将下列文本翻译成英文：\",\n",
    "    \"input\":\"今天天气真好\",\n",
    "    \"output\":\"Today is a nice day！\"\n",
    "}\n",
    "```\n",
    "\n",
    "同时，为使模型能够学习到和预训练不同的范式，在 SFT 的过程中，往往会针对性设置特定格式。例如，LLaMA 的 SFT 格式为：\n",
    "\n",
    "    ### Instruction:\\n{{content}}\\n\\n### Response:\\n\n",
    "\n",
    "其中的 content 即为具体的用户指令，也就是说，对于每一个用户指令，将会嵌入到上文的 content 部分，这里的用户指令不仅指上例中的 “instruction”，而是指令和输入的拼接，即模型可以执行的一条完整指令。例如，针对上例，LLaMA 获得的输入应该是：\n",
    "\n",
    "    ### Instruction:\\n将下列文本翻译成英文：今天天气真好\\n\\n### Response:\\n\n",
    "\n",
    "其需要拟合的输出则是：\n",
    "\n",
    "    ### Instruction:\\n将下列文本翻译成英文：今天天气真好\\n\\n### Response:\\nToday is a nice day！\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54e25489",
   "metadata": {},
   "source": [
    "## 4.2.2 构造微调数据集\n",
    "\n",
    "因此，当我们进行 LLM SFT 以提升 LLM 在指定下游任务的表现时，我们需要将训练数据构造成上述格式，并对数据集进行处理来支持模型微调。接下来，我们以角色扮演任务（要求 LLM 扮演甄嬛，以甄嬛的语气、风格与用户对话）为例，演示如何进行微调数据集构造。\n",
    "\n",
    "我们使用从《甄嬛传》剧本提取出来的对话作为基准来构建训练数据集。提取出来的对话包括上述格式的键：\n",
    "\n",
    "- instruction：即对话的上文；\n",
    "- input：此处置为空；\n",
    "- output：即甄嬛的回复。\n",
    "\n",
    "微调数据集可以在此处下载：https://github.com/KMnO4-zx/huanhuan-chat/blob/master/dataset/train/lora/huanhuan.json\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "52fac949-4150-4091-b0c3-2968ab5e385c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'instruction': '小姐，别的秀女都在求中选，唯有咱们小姐想被撂牌子，菩萨一定记得真真儿的——',\n",
       " 'input': '',\n",
       " 'output': '嘘——都说许愿说破是不灵的。'}"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 加载第三方库\n",
    "from datasets import Dataset\n",
    "import pandas as pd\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM, DataCollatorForSeq2Seq, TrainingArguments, Trainer\n",
    "\n",
    "# 将JSON文件转换为CSV文件\n",
    "df = pd.read_json('./huanhuan.json') # 注意修改\n",
    "ds = Dataset.from_pandas(df)\n",
    "\n",
    "ds[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e93edd90",
   "metadata": {},
   "source": [
    "接下来我们需要定义一个数据处理函数，该函数能够使用 Qwen-3-4B 的 tokenizer（分词器，即将自然语言文本转化为向量）对提供的训练文本进行处理并送到模型中进行学习。\n",
    "\n",
    "如上所说，不同 LLM 存在不同的指令格式，在我们进行训练时，需要遵守 LLM 预定义的指令格式，才能使训练取得较好的效果。我们首先查看一下 Qwen-3-4B 的指令格式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "74ee5a67-2e55-4974-b90e-cbf492de500a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|im_start|>system\n",
      "===system_message_test===<|im_end|>\n",
      "<|im_start|>user\n",
      "===user_message_test===<|im_end|>\n",
      "<|im_start|>assistant\n",
      "<think>\n",
      "\n",
      "</think>\n",
      "\n",
      "===assistant_message_test===<|im_end|>\n",
      "<|im_start|>assistant\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 加载模型 tokenizer \n",
    "tokenizer = AutoTokenizer.from_pretrained('model/Qwen/Qwen3-4B-Instruct-2507', trust_remote=True)\n",
    "\n",
    "# 打印一下 chat template\n",
    "messages = [\n",
    "    {\"role\": \"system\", \"content\": \"===system_message_test===\"},\n",
    "    {\"role\": \"user\", \"content\": \"===user_message_test===\"},\n",
    "    {\"role\": \"assistant\", \"content\": \"===assistant_message_test===\"},\n",
    "]\n",
    "\n",
    "text = tokenizer.apply_chat_template(\n",
    "    messages,\n",
    "    tokenize=False,\n",
    "    add_generation_prompt=True,\n",
    "    enable_thinking=True\n",
    ")\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1cf0242",
   "metadata": {},
   "source": [
    "接着，我们基于上文打印的指令格式，完成数据集处理函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2503a5fa-9621-4495-9035-8e7ef6525691",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def process_func(example):\n",
    "    MAX_LENGTH = 1024 # 设置最大序列长度为1024个token\n",
    "    input_ids, attention_mask, labels = [], [], [] # 初始化返回值\n",
    "    # 适配chat_template\n",
    "    instruction = tokenizer(\n",
    "        f\"<s><|im_start|>system\\n现在你要扮演皇帝身边的女人--甄嬛<|im_end|>\\n\" \n",
    "        f\"<|im_start|>user\\n{example['instruction'] + example['input']}<|im_end|>\\n\"  \n",
    "        f\"<|im_start|>assistant\\n<think>\\n\\n</think>\\n\\n\",  \n",
    "        add_special_tokens=False   \n",
    "    )\n",
    "    response = tokenizer(f\"{example['output']}\", add_special_tokens=False)\n",
    "    # 将instructio部分和response部分的input_ids拼接，并在末尾添加eos token作为标记结束的token\n",
    "    input_ids = instruction[\"input_ids\"] + response[\"input_ids\"] + [tokenizer.pad_token_id]\n",
    "    # 注意力掩码，表示模型需要关注的位置\n",
    "    attention_mask = instruction[\"attention_mask\"] + response[\"attention_mask\"] + [1]\n",
    "    # 对于instruction，使用-100表示这些位置不计算loss（即模型不需要预测这部分）\n",
    "    labels = [-100] * len(instruction[\"input_ids\"]) + response[\"input_ids\"] + [tokenizer.pad_token_id]  \n",
    "    if len(input_ids) > MAX_LENGTH:  # 超出最大序列长度截断\n",
    "        input_ids = input_ids[:MAX_LENGTH]\n",
    "        attention_mask = attention_mask[:MAX_LENGTH]\n",
    "        labels = labels[:MAX_LENGTH]\n",
    "    return {\n",
    "        \"input_ids\": input_ids,\n",
    "        \"attention_mask\": attention_mask,\n",
    "        \"labels\": labels\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "84f870d6-73a9-4b0f-8abf-687b32224ad8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "252cb06d3d1245b5ad739ebe5401264c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3729 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['input_ids', 'attention_mask', 'labels'],\n",
       "    num_rows: 3729\n",
       "})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 使用上文定义的函数对数据集进行处理\n",
    "tokenized_id = ds.map(process_func, remove_columns=ds.column_names)\n",
    "tokenized_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1f7e15a0-4d9a-4935-9861-00cc472654b1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<s><|im_start|>system\n",
      "现在你要扮演皇帝身边的女人--甄嬛<|im_end|>\n",
      "<|im_start|>user\n",
      "小姐，别的秀女都在求中选，唯有咱们小姐想被撂牌子，菩萨一定记得真真儿的——<|im_end|>\n",
      "<|im_start|>assistant\n",
      "<think>\n",
      "\n",
      "</think>\n",
      "\n",
      "嘘——都说许愿说破是不灵的。<|endoftext|>\n"
     ]
    }
   ],
   "source": [
    "# 最终模型的输入\n",
    "print(tokenizer.decode(tokenized_id[0]['input_ids']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "97f16f66-324a-454f-8cc3-ef23b100ecff",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "嘘——都说许愿说破是不灵的。<|endoftext|>\n"
     ]
    }
   ],
   "source": [
    "# 最终模型的输出\n",
    "print(tokenizer.decode(list(filter(lambda x: x != -100, tokenized_id[0][\"labels\"]))))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13d71257-3c1c-4303-8ff8-af161ebc2cf1",
   "metadata": {},
   "source": [
    "## 4.3 高效微调-LoRA "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ec1e9e7",
   "metadata": {},
   "source": [
    "### 4.4.1 什么是高效微调\n",
    "\n",
    "目前，主流的模型微调方法包括全量微调与高效微调。全量微调即在 SFT 过程中会更新模型全部的参数，因此需要较大的算力，成本较为高昂。针对全量微调的昂贵问题，业界提出了高效微调的方法，即通过向模型中插入新的层，在微调时仅更新新层的少量参数等方法来降低微调的算力需求。\n",
    "\n",
    "目前，最主流的高效微调方法是 LoRA 微调。LoRA 微调的基本原理是，向模型中插入一些低秩矩阵层作为 LoRA 微调的目标参数，在微调时仅更新这部分参数。由于 LoRA 微调一般只需要模型在预训练的基础上学会一些较简单的任务，因此即使只更新部分插入参数，仍然能达到较好的微调效果。同时，LoRA 插入的低秩矩阵是对原参数的分解，在推理时，可通过矩阵计算直接将 LoRA 参数合并到原模型，从而避免了推理速度的降低。关于 LoRA 微调的更多原理，可以参考[Happy LLM 中关于 LoRA 微调的部分](https://github.com/datawhalechina/happy-llm/blob/main/docs/chapter6/%E7%AC%AC%E5%85%AD%E7%AB%A0%20%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E6%B5%81%E7%A8%8B%E5%AE%9E%E8%B7%B5.md)。\n",
    "\n",
    "因此，LoRA 成为目前微调 LLM 的主流方法，尤其是对于资源受限、有监督训练数据受限的情况下，LoRA 微调往往会成为 LLM 微调的首选方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "959b9b4a",
   "metadata": {},
   "source": [
    "### 4.4.2 如何进行 LoRA 微调\n",
    "\n",
    "我们使用 peft 库来高效、便捷地实现 LoRA 微调。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2d304ae2-ab60-4080-a80d-19cac2e3ade3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LoraConfig(task_type=<TaskType.CAUSAL_LM: 'CAUSAL_LM'>, peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path=None, revision=None, inference_mode=False, r=8, target_modules={'q_proj', 'v_proj', 'up_proj', 'k_proj', 'down_proj', 'gate_proj', 'o_proj'}, exclude_modules=None, lora_alpha=32, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', trainable_token_indices=None, loftq_config={}, eva_config=None, corda_config=None, use_dora=False, use_qalora=False, qalora_group_size=16, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False), lora_bias=False, target_parameters=None)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 首先配置 LoRA 参数\n",
    "from peft import LoraConfig, TaskType, get_peft_model\n",
    "\n",
    "config = LoraConfig(\n",
    "    task_type=TaskType.CAUSAL_LM, # 任务类型为 CLM，即 SFT 任务的类型\n",
    "    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"], # 目标模块，即需要进行 LoRA 微调的模块\n",
    "    inference_mode=False, # 训练模式\n",
    "    r=8, # Lora 秩，即 LoRA 微调的维度\n",
    "    lora_alpha=32, # Lora alaph，具体作用参见 Lora 原理\n",
    "    lora_dropout=0.1# Dropout 比例\n",
    ")\n",
    "config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2c2489c5-eaab-4e1f-b06a-c3f914b4bf8e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "040627b4aab14b7db79fe7fc07f85864",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "LoraConfig(task_type=<TaskType.CAUSAL_LM: 'CAUSAL_LM'>, peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path='model/Qwen/Qwen3-4B-Instruct-2507', revision=None, inference_mode=False, r=8, target_modules={'q_proj', 'v_proj', 'up_proj', 'k_proj', 'down_proj', 'gate_proj', 'o_proj'}, exclude_modules=None, lora_alpha=32, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', trainable_token_indices=None, loftq_config={}, eva_config=None, corda_config=None, use_dora=False, use_qalora=False, qalora_group_size=16, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False), lora_bias=False, target_parameters=None)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "# 加载基座模型\n",
    "model = AutoModelForCausalLM.from_pretrained('model/Qwen/Qwen3-4B-Instruct-2507', device_map=\"auto\",torch_dtype=torch.bfloat16)\n",
    "# 开启模型梯度检查点能降低训练的显存占用\n",
    "model.enable_input_require_grads()\n",
    "# 通过下列代码即可向模型中添加 LoRA 模块\n",
    "model = get_peft_model(model, config)\n",
    "config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ebf5482b-fab9-4eb3-ad88-c116def4be12",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable params: 16,515,072 || all params: 4,038,983,168 || trainable%: 0.4089\n"
     ]
    }
   ],
   "source": [
    "# 查看 lora 微调的模型参数\n",
    "model.print_trainable_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca9e0dc4-7a9c-406c-a71e-36e162ed6143",
   "metadata": {},
   "source": [
    "本文使用 Swanlab 进行训练的监测，可以便捷地查看训练的 loss 情况、GPU 利用率等。通过以下命令安装 swanlab 第三方库："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cc115f0-d3a2-467e-91e0-5648499c4623",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install swanlab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c6edcf02-f7eb-47cf-9c4c-712955869a0c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e522f4f9a27f49a78b9acc312ca6bafb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 配置 swanlab\n",
    "import swanlab\n",
    "from swanlab.integration.transformers import SwanLabCallback\n",
    "\n",
    "swanlab.login(api_key='your api key', save=False)\n",
    "\n",
    "# 实例化SwanLabCallback\n",
    "swanlab_callback = SwanLabCallback(\n",
    "    project=\"Qwen3-4B-lora\", \n",
    "    experiment_name=\"Qwen3-4B-experiment\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca99384a-3414-4a1c-8c44-dbbee55eded0",
   "metadata": {},
   "source": [
    "swanlab 的 api key 可以通过登录官网注册账号获得：https://swanlab.cn/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "73a211a5-3df6-4e25-9e78-44d90a2dc0da",
   "metadata": {},
   "outputs": [],
   "source": [
    "from swanlab.integration.transformers import SwanLabCallback\n",
    "\n",
    "# 配置训练参数\n",
    "args = TrainingArguments(\n",
    "    output_dir=\"./output/Qwen3_4B_lora\", # 输出目录\n",
    "    per_device_train_batch_size=16, # 每个设备上的训练批量大小\n",
    "    gradient_accumulation_steps=2, # 梯度累积步数\n",
    "    logging_steps=10, # 每10步打印一次日志\n",
    "    num_train_epochs=3, # 训练轮数\n",
    "    save_steps=100, # 每100步保存一次模型\n",
    "    learning_rate=1e-4, # 学习率\n",
    "    save_on_each_node=True, # 是否在每个节点上保存模型\n",
    "    gradient_checkpointing=True, # 是否使用梯度检查点\n",
    "    report_to=\"none\", # 不使用任何报告工具\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "00b997b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 然后使用 trainer 训练即可\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=args,\n",
    "    train_dataset=tokenized_id,\n",
    "    data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True),\n",
    "    callbacks=[swanlab_callback]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9faedcb1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4184d03ec459450da3b23bd7e47f5e32",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">swanlab</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">:</span> Tracking run with swanlab version <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.6</span>.<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;34mswanlab\u001b[0m\u001b[1;39m:\u001b[0m Tracking run with swanlab version \u001b[1;36m0.6\u001b[0m.\u001b[1;36m8\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">swanlab</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">:</span> Run data will be saved locally in <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">/root/autodl-tmp/swanlog/run-20250822_233548-2r8h8qplwecjhk7mtgqts</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;34mswanlab\u001b[0m\u001b[1;39m:\u001b[0m Run data will be saved locally in \u001b[1;35m/root/autodl-tmp/swanlog/run-20250822_233548-2r8h8qplwecjhk7mtgqts\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">swanlab</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">:</span><span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\"> </span>👋 Hi <span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">logan_zou</span>,welcome to swanlab!\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;34mswanlab\u001b[0m\u001b[1;39m:\u001b[0m\u001b[1;34m \u001b[0m👋 Hi \u001b[1;39mlogan_zou\u001b[0m,welcome to swanlab!\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">swanlab</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">:</span> Syncing run <span style=\"color: #808000; text-decoration-color: #808000\">Qwen3-4B-experiment</span> to the cloud\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;34mswanlab\u001b[0m\u001b[1;39m:\u001b[0m Syncing run \u001b[33mQwen3-4B-experiment\u001b[0m to the cloud\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">swanlab</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">:</span> 🏠 View project at <span style=\"color: #000080; text-decoration-color: #000080; text-decoration: underline\">https://swanlab.cn/@logan_zou/Qwen3-4B-lora</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;34mswanlab\u001b[0m\u001b[1;39m:\u001b[0m 🏠 View project at \u001b[4;34mhttps://swanlab.cn/@logan_zou/Qwen3-4B-lora\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">swanlab</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">:</span> 🚀 View run at <span style=\"color: #000080; text-decoration-color: #000080; text-decoration: underline\">https://swanlab.cn/@logan_zou/Qwen3-4B-lora/runs/2r8h8qplwecjhk7mtgqts</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;34mswanlab\u001b[0m\u001b[1;39m:\u001b[0m 🚀 View run at \u001b[4;34mhttps://swanlab.cn/@logan_zou/Qwen3-4B-lora/runs/2r8h8qplwecjhk7mtgqts\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<!DOCTYPE html>\n",
       "<html lang=\"en\">\n",
       "<head>\n",
       "    <meta charset=\"UTF-8\">\n",
       "    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n",
       "    <title>Show Iframe</title>\n",
       "    \n",
       "        <script>\n",
       "            function showIframe() {\n",
       "                var iframeHtml = '<iframe src=\"https://swanlab.cn/@logan_zou/Qwen3-4B-lora/runs/2r8h8qplwecjhk7mtgqts\" width=100% height=\"600\" frameborder=\"no\"></iframe>';\n",
       "                document.getElementById('iframeContainer').innerHTML = iframeHtml;\n",
       "            }\n",
       "        </script>\n",
       "        \n",
       "</head>\n",
       "<body>\n",
       "    <style>\n",
       "        .interactive-button {\n",
       "            display: flex;\n",
       "            align-items: center;\n",
       "            height: 36px;\n",
       "            border: 0px;\n",
       "            background-color: #2c8f63;\n",
       "            color: white;\n",
       "            padding: 10px 20px;\n",
       "            transition: background-color 0.3s, transform 0.2s;\n",
       "        }\n",
       "\n",
       "        .interactive-button:hover {\n",
       "            background-color: #5cab87;\n",
       "            cursor: pointer;\n",
       "        }\n",
       "\n",
       "        .interactive-button:active { background-color: #217952; transform: scale(0.96); } </style> <br> <button \n",
       "        onclick=\"showIframe()\" class=\"interactive-button\"> <svg style=\"height: 16px; margin-right: 8px;\" viewBox=\"0 0 \n",
       "        46 46\" fill=\"none\"> <path d=\"M10.8439 21.1974C10.6414 21.2854 10.4477 21.3925 10.2655 21.5173L10.2069 \n",
       "        21.5652C10.1839 21.58 10.1625 21.5969 10.1429 21.6159C6.29135 24.6118 4.22831 29.4416 5.32646 34.282C5.94656 \n",
       "        37.0577 7.50461 39.5348 9.73801 41.2958C11.9714 43.0568 14.7436 43.994 17.5874 43.9495H18.0219C19.8864 \n",
       "        43.8697 21.7087 43.3694 23.3526 42.486C24.9964 41.6026 26.4193 40.3589 27.5147 38.848C28.61 37.3371 29.3496 \n",
       "        35.598 29.678 33.761C30.0065 31.9239 29.9153 30.0363 29.4112 28.2395C28.9181 26.4723 27.8919 24.8437 26.9937 \n",
       "        23.2551C25.4158 20.4653 23.8343 17.6764 22.2492 14.8884C21.7801 14.0647 21.3057 13.2465 20.8419 \n",
       "        12.4228C20.2315 11.3353 19.2746 10.1519 19.224 8.86183C19.1733 7.57176 20.2235 6.32701 21.5082 \n",
       "        6.07912C23.9284 5.61801 25.0639 8.24078 25.0693 8.23812C25.363 8.94035 25.9123 9.50489 26.6063 \n",
       "        9.81764C27.3002 10.1304 28.087 10.168 28.8077 9.92298C29.5283 9.67791 30.1291 9.1684 30.4885 8.49743C30.8479 \n",
       "        7.82646 30.9392 7.04405 30.7439 6.30835C30.1514 4.37314 28.9133 2.69953 27.2363 1.56656C25.7615 0.511704 \n",
       "        23.9847 -0.0372109 22.1719 0.00195984C20.9049 0.00893199 19.6532 0.27989 18.4967 0.797557C17.3402 1.31522 \n",
       "        16.3043 2.06823 15.4551 3.00856C14.49 4.08707 13.7984 5.38193 13.4389 6.78385C13.0794 8.18576 13.0624 9.6536 \n",
       "        13.3894 11.0635C13.52 11.593 13.6984 12.1095 13.9225 12.6067C14.5595 14.0514 15.4951 15.3681 16.284 \n",
       "        16.7355C17.2525 18.4147 18.2209 20.0948 19.1893 21.7758C20.1578 23.4568 21.1351 25.1449 22.1213 \n",
       "        26.8401C22.9209 28.2421 23.7925 29.4682 23.8805 31.1528C23.9175 32.0513 23.7682 32.9479 23.4419 \n",
       "        33.7859C23.1156 34.6239 22.6194 35.3854 21.9845 36.0223C21.3496 36.6592 20.5897 37.1578 19.7527 \n",
       "        37.4868C18.9157 37.8157 18.0196 37.9678 17.121 37.9336C14.0024 37.7923 11.6488 35.4814 11.1744 32.4588C10.58 \n",
       "        28.6419 13.552 26.5469 13.552 26.5469C14.1782 26.1785 14.6497 25.5955 14.8791 24.906C15.1084 24.2166 15.0801 \n",
       "        23.4673 14.7993 22.7971C14.5186 22.127 14.0044 21.5813 13.3521 21.2611C12.6998 20.941 11.9536 20.8682 11.2517 \n",
       "        21.0561C11.1174 21.0939 10.9856 21.1402 10.8572 21.1947\" fill=\"white\" /> <path d=\"M42.8101 31.5968C42.8109 \n",
       "        30.5198 42.7218 29.4445 42.5435 28.3823C42.2663 26.7069 41.7464 25.0808 41.0002 23.5552C40.5524 22.6463 \n",
       "        39.9874 21.7374 39.1024 21.2417C38.6593 20.9919 38.1589 20.8617 37.6502 20.8639C37.1416 20.8661 36.6423 \n",
       "        21.0006 36.2013 21.2541C35.7604 21.5077 35.393 21.8716 35.1352 22.3101C34.8775 22.7485 34.7382 23.2466 \n",
       "        34.7312 23.7552C34.7072 24.8773 35.3149 25.8875 35.768 26.9217C36.5212 28.6453 36.8623 30.5208 36.7642 \n",
       "        32.3993C36.6661 34.2777 36.1315 36.1075 35.2029 37.7433C35.146 37.8404 35.0952 37.941 35.051 38.0445C34.8623 \n",
       "        38.4842 34.7635 38.9573 34.7605 39.4358C34.7802 40.1222 35.0356 40.7808 35.4835 41.3011C35.9315 41.8214 \n",
       "        36.5449 42.1717 37.2207 42.2932C38.8759 42.589 40.1899 41.347 40.8856 39.9609C42.1643 37.3589 42.823 34.4961 \n",
       "        42.8101 31.5968Z\" fill=\"white\" /> <path d=\"M28.2309 11.8938C28.1761 11.9043 28.1218 11.9176 28.0683 \n",
       "        11.9338C27.9593 11.9642 27.8611 12.0249 27.7851 12.1088C27.7091 12.1928 27.6584 12.2965 27.6389 \n",
       "        12.408C27.6193 12.5195 27.6318 12.6343 27.6748 12.7391C27.7178 12.8438 27.7895 12.9343 27.8818 \n",
       "        12.9999C29.2375 14.0252 30.3809 15.3043 31.2482 16.7662C31.4838 17.1677 31.6888 17.5865 31.8612 \n",
       "        18.0189C32.0052 18.3921 32.1971 18.8799 32.6822 18.8532C33.0607 18.8346 33.2153 18.512 33.3192 \n",
       "        18.1895C33.8137 16.5125 33.9678 14.7534 33.7723 13.0159C33.6331 12.0693 33.4155 11.1359 33.122 \n",
       "        10.2252C33.0775 10.0047 32.9744 9.80029 32.8235 9.6335C32.7273 9.54627 32.6054 9.49262 32.4761 9.4806C32.3468 \n",
       "        9.46859 32.2171 9.49886 32.1065 9.56687C32.0016 9.65188 31.9115 9.75365 31.8399 9.86806C31.3956 10.4658 \n",
       "        30.825 10.9581 30.1687 11.3101C29.8377 11.4861 29.4893 11.6272 29.1292 11.7312C28.828 11.8192 28.5215 11.8325 \n",
       "        28.2309 11.8938Z\" fill=\"white\" /> </svg> Display SwanLab Board </button> <br> <div \n",
       "        id=\"iframeContainer\"></div> </body> </html>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='351' max='351' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [351/351 09:58, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>4.089600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>3.294300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30</td>\n",
       "      <td>3.182500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>3.205600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>3.196200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>60</td>\n",
       "      <td>3.151000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>70</td>\n",
       "      <td>3.066400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>80</td>\n",
       "      <td>3.130000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>90</td>\n",
       "      <td>3.089900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>3.081900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>110</td>\n",
       "      <td>3.027100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>120</td>\n",
       "      <td>3.079900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>130</td>\n",
       "      <td>2.831500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>140</td>\n",
       "      <td>2.856900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150</td>\n",
       "      <td>2.862200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>160</td>\n",
       "      <td>2.807000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>170</td>\n",
       "      <td>2.827600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>180</td>\n",
       "      <td>2.848600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>190</td>\n",
       "      <td>2.851700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>2.792800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>210</td>\n",
       "      <td>2.829700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>220</td>\n",
       "      <td>2.827600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>230</td>\n",
       "      <td>2.802900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>240</td>\n",
       "      <td>2.642300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>250</td>\n",
       "      <td>2.581000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>260</td>\n",
       "      <td>2.602000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>270</td>\n",
       "      <td>2.605900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>280</td>\n",
       "      <td>2.686700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>290</td>\n",
       "      <td>2.564800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>300</td>\n",
       "      <td>2.542900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>310</td>\n",
       "      <td>2.558300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>320</td>\n",
       "      <td>2.611400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>330</td>\n",
       "      <td>2.637400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>340</td>\n",
       "      <td>2.615000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>350</td>\n",
       "      <td>2.516300</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=351, training_loss=2.8820284526911895, metrics={'train_runtime': 604.2405, 'train_samples_per_second': 18.514, 'train_steps_per_second': 0.581, 'total_flos': 3.2568125184497664e+16, 'train_loss': 2.8820284526911895, 'epoch': 3.0})"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'RemoteDisconnected('Remote end closed connection without response')': /api/house/metrics\n"
     ]
    }
   ],
   "source": [
    "# 开始训练\n",
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bbc0c69",
   "metadata": {},
   "source": [
    "LoRA 微调仅保存微调后的 LoRA 参数，因此推理微调模型需要加载 LoRA 参数并合并："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "bd2a415a-a9ad-49ea-877f-243558a83bfc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "747f3316908e4e48a3b7ef42f11433c8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
    "import torch\n",
    "from peft import PeftModel\n",
    "\n",
    "mode_path = 'model/Qwen/Qwen3-4B-Instruct-2507'# 基座模型参数路径\n",
    "lora_path = './output/Qwen3_4B_lora/checkpoint-351' # 这里改成你的 lora 输出对应 checkpoint 地址\n",
    "\n",
    "# 加载tokenizer\n",
    "tokenizer = AutoTokenizer.from_pretrained(mode_path)\n",
    "\n",
    "# 加载模型\n",
    "model = AutoModelForCausalLM.from_pretrained(mode_path, device_map=\"auto\",torch_dtype=torch.bfloat16, trust_remote_code=True)\n",
    "\n",
    "# 加载lora权重\n",
    "model = PeftModel.from_pretrained(model, model_id=lora_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2dff0799",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "我是甄嬛，家父是大理寺少卿甄远道。\n"
     ]
    }
   ],
   "source": [
    "prompt = \"你是谁？\"\n",
    "inputs = tokenizer.apply_chat_template(\n",
    "                                    [{\"role\": \"user\", \"content\": \"假设你是皇帝身边的女人--甄嬛。\"},{\"role\": \"user\", \"content\": prompt}],\n",
    "                                    add_generation_prompt=True,\n",
    "                                    tokenize=True,\n",
    "                                    return_tensors=\"pt\",\n",
    "                                    return_dict=True,\n",
    "                                    enable_thinking=False\n",
    "                                )\n",
    "inputs = inputs.to(\"cuda\")\n",
    "\n",
    "\n",
    "gen_kwargs = {\"max_length\": 2500, \"do_sample\": True, \"top_k\": 1}\n",
    "with torch.no_grad():\n",
    "    outputs = model.generate(**inputs, **gen_kwargs)\n",
    "    outputs = outputs[:, inputs['input_ids'].shape[1]:]\n",
    "    print(tokenizer.decode(outputs[0], skip_special_tokens=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7ba8ac0",
   "metadata": {},
   "source": [
    "# 课后作业\n",
    "\n",
    "请大家选择一个 NLP 经典任务（例如情感分类、命名实体识别等），收集该任务的经典训练数据对 Qwen3-4B 进行微调，并评估微调后模型在该任务上的效果\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9993df9a-8d87-4e6a-a92e-2de994b17d84",
   "metadata": {},
   "source": [
    "# 后续学习建议"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8217f68a-6697-4380-b7e3-8a2f0328253a",
   "metadata": {},
   "source": [
    "**最重要的事——确定目标**\n",
    "\n",
    "针对学完了本次课程的同学，可以根据个人背景与学习目标对应选择自己的后续学习方向：\n",
    "\n",
    "- 对非 CS/DS 专业同学：\n",
    "  - 目标是**使用** LLM 辅助本专业研究，则掌握上述技能已经能够帮助你使用 LLM 赋能专业研究，建议选择专业研究问题，思考如何使用 LLM 赋能并投入实际实践；\n",
    "  - 目标是**深入探索 AI+X**道路，则建议进一步学习 LLM 底层原理，掌握更系统、全面的 LLM 知识，并向上拓展学习 LLM 更多的应用形式（例如 RAG、Agents 等），通过不断探索找到专业与 LLM 深度融合的路径：\n",
    "    - 推荐学习：[Happy LLM](https://github.com/datawhalechina/happy-llm)\n",
    "- 对 CS/DS 专业的初学同学：\n",
    "  - 目标是深入学习 LLM 且计划**未来从事相关职业**的同学，建议从 ML、NLP 等方向的底层原理开始学习，同步进行基础知识学习及基于本教程展开一些 LLM 实践（做一些 LLM 项目）；\n",
    "    - 推荐学习：[南瓜书](https://github.com/datawhalechina/pumpkin-book)、[李宏毅深度学习教程](https://github.com/datawhalechina/leedl-tutorial)、[Happy LLM](https://github.com/datawhalechina/happy-llm)、[Self LLM](https://github.com/datawhalechina/self-llm);\n",
    "  - 目标是快速掌握 LLM 的同学，建议进一步学习 LLM 底层原理，掌握更系统、全面的 LLM 知识，并向上拓展学习 LLM 更多的应用形式（例如 RAG、Agents 等）:\n",
    "    - 推荐学习：[Happy LLM](https://github.com/datawhalechina/happy-llm)、[Self LLM](https://github.com/datawhalechina/self-llm);"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
